Gradient-based Reinforcement Planning in Policy-Search Methods

We introduce a learning method called ``gradient-based reinforcement planning'' (GREP). Unlike traditional DP methods that improve their policy backwards in time, GREP is a gradient-based method that plans ahead and improves its policy before it actually acts in the environment. We derive formulas for the exact policy gradient that maximizes the expected future reward and confirm our ideas with numerical experiments.Comment: This is an extended version of the paper presented at the EWRL 2001 in Utrecht (The Netherlands

Bayesian DNA copy number analysis

BACKGROUND: Some diseases, like tumors, can be related to chromosomal aberrations, leading to changes of DNA copy number. The copy number of an aberrant genome can be represented as a piecewise constant function, since it can exhibit regions of deletions or gains. Instead, in a healthy cell the copy number is two because we inherit one copy of each chromosome from each our parents. Bayesian Piecewise Constant Regression (BPCR) is a Bayesian regression method for data that are noisy observations of a piecewise constant function. The method estimates the unknown segment number, the endpoints of the segments and the value of the segment levels of the underlying piecewise constant function. The Bayesian Regression Curve (BRC) estimates the same data with a smoothing curve. However, in the original formulation, some estimators failed to properly determine the corresponding parameters. For example, the boundary estimator did not take into account the dependency among the boundaries and succeeded in estimating more than one breakpoint at the same position, losing segments. RESULTS: We derived an improved version of the BPCR (called mBPCR) and BRC, changing the segment number estimator and the boundary estimator to enhance the fitting procedure. We also proposed an alternative estimator of the variance of the segment levels, which is useful in case of data with high noise. Using artificial data, we compared the original and the modified version of BPCR and BRC with other regression methods, showing that our improved version of BPCR generally outperformed all the others. Similar results were also observed on real data. CONCLUSION: We propose an improved method for DNA copy number estimation, mBPCR, which performed very well compared to previously published algorithms. In particular, mBPCR was more powerful in the detection of the true position of the breakpoints and of small aberrations in very noisy data. Hence, from a biological point of view, our method can be very useful, for example, to find targets of genomic aberrations in clinical cancer samples

An integrated Bayesian analysis of LOH and copy number data

BACKGROUND Cancer and other disorders are due to genomic lesions. SNP-microarrays are able to measure simultaneously both genotype and copy number (CN) at several Single Nucleotide Polymorphisms (SNPs) along the genome. CN is defined as the number of DNA copies, and the normal is two, since we have two copies of each chromosome. The genotype of a SNP is the status given by the nucleotides (alleles) which are present on the two copies of DNA. It is defined homozygous or heterozygous if the two alleles are the same or if they differ, respectively. Loss of heterozygosity (LOH) is the loss of the heterozygous status due to genomic events. Combining CN and LOH data, it is possible to better identify different types of genomic aberrations. For example, a long sequence of homozygous SNPs might be caused by either the physical loss of one copy or a uniparental disomy event (UPD), i.e. each SNP has two identical nucleotides both derived from only one parent. In this situation, the knowledge of the CN can help in distinguishing between these two events. RESULTS To better identify genomic aberrations, we propose a method (called gBPCR) which infers the type of aberration occurred, taking into account all the possible influence in the microarray detection of the homozygosity status of the SNPs, resulting from an altered CN level. Namely, we model the distributions of the detected genotype, given a specific genomic alteration and we estimate the parameters involved on public reference datasets. The estimation is performed similarly to the modified Bayesian Piecewise Constant Regression, but with improved estimators for the detection of the breakpoints.Using artificial and real data, we evaluate the quality of the estimation of gBPCR and we also show that it outperforms other well-known methods for LOH estimation. CONCLUSIONS We propose a method (gBPCR) for the estimation of both LOH and CN aberrations, improving their estimation by integrating both types of data and accounting for their relationships. Moreover, gBPCR performed very well in comparison with other methods for LOH estimation and the estimated CN lesions on real data have been validated with another technique.This work was supported by Swiss National Science Foundation (grants 205321-112430, 205320-121886/1); Oncosuisse grants OCS-1939-8-2006 and OCS - 02296-08-2008; Cantone Ticino ("Computational life science/Ticino in rete” program); Fondazione per la Ricerca e la Cura sui Linfomi (Lugano, Switzerland)

Market-based reinforcement learning in partially observable worlds

Unlike traditional reinforcement learning (RL), market-based RL is in principle applicable to worlds described by partially observable Markov Decision Processes (POMDPs), where an agent needs to learn short-term memories of relevant previous events in order to execute optimal actions. Most previous work, however, has focused on reactive settings (MDPs) instead of POMDPs. Here we reimplement a recent approach to market-based RL and for the first time evaluate it in a toy POMDP setting.This work was supported by SNF grants 21-55409.98 and 2000-61847.0

PCSF: An R-package for network-based interpretation of high-throughput data

With the recent technological developments a vast amount of high-throughput data has been profiled to understand the mechanism of complex diseases. The current bioinformatics challenge is to interpret the data and underlying biology, where efficient algorithms for analyzing heterogeneous high-throughput data using biological networks are becoming increasingly valuable. In this paper, we propose a software package based on the Prize-collecting Steiner Forest graph optimization approach. The PCSF package performs fast and user-friendly network analysis of high-throughput data by mapping the data onto a biological networks such as protein-protein interaction, gene-gene interaction or any other correlation or coexpression based networks. Using the interaction networks as a template, it determines high-confidence subnetworks relevant to the data, which potentially leads to predictions of functional units. It also interactively visualizes the resulting subnetwork with functional enrichment analysis

Novel GC-rich DNA-binding compound produced by a genetically engineered mutant of the mithramycin producer Streptomyces argillaceus exhibits improved transcriptional repressor activity: implications for cancer therapy

The aureolic acid antibiotic mithramycin (MTM) binds selectively to GC-rich DNA sequences and blocks preferentially binding of proteins, like Sp1 transcription factors, to GC-rich elements in gene promoters. Genetic approaches can be applied to alter the MTM biosynthetic pathway in the producing microorganism and obtain new products with improved pharmacological properties. Here, we report on a new analog, MTM SDK, obtained by targeted gene inactivation of the ketoreductase MtmW catalyzing the last step in MTM biosynthesis. SDK exhibited greater activity as transcriptional inhibitor compared to MTM. SDK was a potent inhibitor of Sp1-dependent reporter activity and interfered minimally with reporters of other transcription factors, indicating that it retained a high degree of selectivity toward GC-rich DNA-binding transcription factors. RT–PCR and microarray analysis showed that SDK repressed transcription of multiple genes implicated in critical aspects of cancer development and progression, including cell cycle, apoptosis, migration, invasion and angiogenesis, consistent with the pleiotropic role of Sp1 family transcription factors. SDK inhibited proliferation and was a potent inducer of apoptosis in ovarian cancer cells while it had minimal effects on viability of normal cells. The new MTM derivative SDK could be an effective agent for treatment of cancer and other diseases with abnormal expression or activity of GC-rich DNA-binding transcription factors

Systems analyses of the Fabry kidney transcriptome and its response to enzyme replacement therapy identified and cross-validated enzyme replacement therapy-resistant targets amenable to drug repurposing

Fabry disease is a rare disorder caused by variations in the alpha-galactosidase gene. To a degree, Fabry disease is manageable via enzyme replacement therapy (ERT). By understanding the molecular basis of Fabry nephropathy (FN) and ERT's long-term impact, here we aimed to provide a framework for selection of potential disease biomarkers and drug targets. We obtained biopsies from eight control individuals and two independent FN cohorts comprising 16 individuals taken prior to and after up to ten years of ERT, and performed RNAseq analysis. Combining pathway-centered analyses with network-science allowed computation of transcriptional landscapes from four nephron compartments and their integration with existing proteome and drug-target interactome data. Comparing these transcriptional landscapes revealed high inter-cohort heterogeneity. Kidney compartment transcriptional landscapes comprehensively reflected differences in FN cohort characteristics. With exception of a few aspects, in particular arteries, early ERT in patients with classical Fabry could lastingly revert FN gene expression patterns to closely match that of control individuals. Pathways nonetheless consistently altered in both FN cohorts pre-ERT were mostly in glomeruli and arteries and related to the same biological themes. While keratinization-related processes in glomeruli were sensitive to ERT, a majority of alterations, such as transporter activity and responses to stimuli, remained dysregulated or reemerged despite ERT. Inferring an ERT-resistant genetic module of expressed genes identified 69 drugs for potential repurposing matching the proteins encoded by 12 genes. Thus, we identified and cross-validated ERT-resistant gene product modules that, when leveraged with external data, allowed estimating their suitability as biomarkers to potentially track disease course or treatment efficacy and potential targets for adjunct pharmaceutical treatment

Apyrase-mediated amplification of secretory IgA promotes intestinal homeostasis.

Secretory immunoglobulin A (SIgA) interaction with commensal bacteria conditions microbiota composition and function. However, mechanisms regulating reciprocal control of microbiota and SIgA are not defined. Bacteria-derived adenosine triphosphate (ATP) limits T follicular helper (Tfh) cells in the Peyer's patches (PPs) via P2X7 receptor (P2X7R) and thereby SIgA generation. Here we show that hydrolysis of extracellular ATP (eATP) by apyrase results in amplification of the SIgA repertoire. The enhanced breadth of SIgA in mice colonized with apyrase-releasing Escherichia coli influences topographical distribution of bacteria and expression of genes involved in metabolic versus immune functions in the intestinal epithelium. SIgA-mediated conditioning of bacteria and enterocyte function is reflected by differences in nutrient absorption in mice colonized with apyrase-expressing bacteria. Apyrase-induced SIgA improves intestinal homeostasis and attenuates barrier impairment and susceptibility to infection by enteric pathogens in antibiotic-induced dysbiosis. Therefore, amplification of SIgA by apyrase can be leveraged to restore intestinal fitness in dysbiotic conditions

Bayesian DNA copy number analysis

<p>Abstract</p> <p>Background</p> <p>Some diseases, like tumors, can be related to chromosomal aberrations, leading to changes of DNA copy number. The copy number of an aberrant genome can be represented as a piecewise constant function, since it can exhibit regions of deletions or gains. Instead, in a healthy cell the copy number is two because we inherit one copy of each chromosome from each our parents.</p> <p>Bayesian Piecewise Constant Regression (BPCR) is a Bayesian regression method for data that are noisy observations of a piecewise constant function. The method estimates the unknown segment number, the endpoints of the segments and the value of the segment levels of the underlying piecewise constant function. The Bayesian Regression Curve (BRC) estimates the same data with a smoothing curve. However, in the original formulation, some estimators failed to properly determine the corresponding parameters. For example, the boundary estimator did not take into account the dependency among the boundaries and succeeded in estimating more than one breakpoint at the same position, losing segments.</p> <p>Results</p> <p>We derived an improved version of the BPCR (called mBPCR) and BRC, changing the segment number estimator and the boundary estimator to enhance the fitting procedure. We also proposed an alternative estimator of the variance of the segment levels, which is useful in case of data with high noise. Using artificial data, we compared the original and the modified version of BPCR and BRC with other regression methods, showing that our improved version of BPCR generally outperformed all the others. Similar results were also observed on real data.</p> <p>Conclusion</p> <p>We propose an improved method for DNA copy number estimation, mBPCR, which performed very well compared to previously published algorithms. In particular, mBPCR was more powerful in the detection of the true position of the breakpoints and of small aberrations in very noisy data. Hence, from a biological point of view, our method can be very useful, for example, to find targets of genomic aberrations in clinical cancer samples.</p